A.2 (cyclic,*) Distribution a Mpi-like Code for Gaussian Elimination A.1 (block,*) Distribution 6 Concluding Remarks Acknowledgement

(j,k) = B(j,k)-B(j,lindex(2)) * B(lindex(1),k) /B(lindex(1),lindex(2)) END DO END DO DO i=n, 1,-1 gindex(1)=i gindex(2)=i CALL GLOBAL-TO-LOCAL(B,gindex,lindex,local) IF (local .EQ. true) THEN x(xindx(i)) = B(lindex(1),UBOUND(B,2)) / B(lindex(1),lindex(2)) proc=i mod p MPI-BCAST(x(indx(i)),tag 3 ,all,proc) DO j = lindex(1)-1, LBOUND(B,1),-1 B(j,UBOUND(B,2)) = B(j,UBOUND(B,2))-B(j,lindex(2)) * x(xindx(i)) END DO ENDIF END DO ! Linear system solver library in MPI-code with (cyclic,*) decomposition SUBROUTINE LINSYS-CYST(B,x) !HPF$ LOCAL REAL INTENT (IN) :: B(:,:) REAL INTENT (OUT) :: x(:) INTEGER xindx(UBOUND(B,2)-1) REAL temp(UBOUND(B,2)-1) INTEGER gindex(2), lindex(2) DO i = 1, n xindx(i) = i END DO DO i = 1, n gindex(1)=i gindex(2)=i CALL GLOBAL-TO-LOCAL(B,gindex,lindex,local) IF (local .EQ. true) THEN maxloc = MAXLOC(B(lindex(1),lindex(2):UBOUND(B,2)-1)) ENDIF proc=m mod p MPI-BCAST(maxloc,tag 1 ,all,proc) maxval = B(lindex(1),maxloc) temp = B(:,maxloc) B(:,maxloc) = B(:,lindex(2)) B(:,lindex(2)) = temp tempx = xindx(maxloc) xindx(maxloc) = xindx(i) xindx(i) = tempx This section shows the MPI-like code of two distributions: (block,*) and (cyclic,*) based on Gaussian elimination and backward substitution. Both implementations are coded in the form of F90/HPF extrinsic. ! Linear system solver library in MPI-code with (block,*) decomposition SUBROUTINE LINSYS-BLST(B,x) !HPF$ LOCAL REAL INTENT (IN) :: B(:,:) REAL INTENT (OUT) :: x(:) INTEGER xindx(UBOUND(B,2)-1) REAL temp(UBOUND(B,2)-1) INTEGER gindex(2), lindex(2) DO i = 1, n xindx(i) = i END DO DO i = 1, n gindex(1)=i gindex(2)=i CALL GLOBAL-TO-LOCAL(B,gindex,lindex,local) IF (local .EQ. true) THEN maxloc = MAXLOC(B(lindex(1),lindex(2):UBOUND(B,2)-1)) ENDIF proc=b i d n p e c MPI-BCAST(maxloc,tag 1 ,all,proc) maxval = B(lindex(1),maxloc) temp = B(:,maxloc) B(:,maxloc) = B(:,lindex(2)) B(:,lindex(2)) = temp tempx = xindx(maxloc) xindx(maxloc) = xindx(i) xindx(i) = tempx MPI-BCAST(B(lindex(1),lindex(2):UBOUND(B,2)), tag 2 ,all,proc) DO j=lindex(1)+1, UBOUND(B,1) DO k=lindex(2)+1, UBOUND(B,2) B(j,k) = B(j,k)-B(j,lindex(2)) * B(lindex(1),k) /B(lindex(1),lindex(2)) END DO END DO DO i=n, 1,-1 gindex(1)=i gindex(2)=i CALL GLOBAL-TO-LOCAL(B,gindex,lindex,local) IF (local .EQ. true) THEN x(xindx(i)) = B(lindex(1),UBOUND(B,2)) / B(lindex(1),lindex(2)) 27 5] \EEcient implementation of barrier synchronization in wormhole-routed hypercube multicomputers," Providing a rich set of libraries to programmers in MPCs is a highly demanded, but not a trivial, task. This paper discussed four major features: scalability, portability, recompilation, and exibil-ity, required in the design of scalable libraries. We advocate a layered structure of libraries to meet diierent demands and requirements from diierent perspectives. In general, the higher two layers provide more exibility and the lower two layers provide better performance. However, as compiler technologies mature, the higher layers may also provide better performance. Both the number of processors and the problem size have a great impact on the library …